Fluid Motion Recovery by Coupling Dense and Parametric Vector Fields

In this paper we address the problem of estimating and analyzing the motion in image sequences that involve fluid phenomena. In this context standard motion estimation techniques are not well adapted and more dedicated approaches have to be designed. In this prospect, we propose to estimate in a joint and cooperative way a dense motion field and a peculiar parametric representation of the flow. The parametric model issues from an extension of Rankine vortex model and includes a laminar flow field. Dense and parametric fields are estimated by minimizing a robust global objective function thanks to a specific alternate scheme. The method has been validated on different kinds of meteorological image sequences. 1 Background: Fluid Motion Estimation In a number of domains, image sequences that involve fluid phenomena, have to be analyzed: In environmental sciences (oceanography, meteorology, climatology, etc.), ocean and atmosphere evolutions are observed via satellite sensors [5, 9]; In medical imaging, blood flow can be monitored by angiography [14]; In the field of fluid mechanics, aeroand hydro-dynamics experiments now routinely produce lots of video data [7, 10, 15]. In all these domains of applications, camera offers in a versatile and non-intrusive way, huge amounts of spatio-temporal data, as opposed to in situ measurement techniques that are often complex, very specific, intrusive, and that only provide with sparse data. With these latter techniques, however, sought quantities are directly measured with dedicated probes, whereas, within image sequences, the relevant information has to be extracted from the luminance data. The analysis of motion in such sequences is particularly challenging due to the great deal of spatial and temporal distortions that luminance patterns exhibit in imaged fluid phenomena. Standard techniques from Computer Vision, originally designed for quasi-rigid motions with stable salient features, are not well adapted in this context. The design of alternate approaches dedicated to fluid motion thus constitutes a widely open domain of research. Our work is a contribution in this direction. As in standard motion analysis, two types of motion information can be sought. First, dense velocity (or displacement) fields [5, 9] constitute precious sources of information which can serve either as validation basis, or as input data for numerical models (e.g., in short-term weather prediction). They are also used for visualization purposes, and allow to compute other quantities of interest, such as the vorticity of the flow [15]. Second, some salient structures may be sought. Vortices [14, 16], and more generally singular points [5, 7, 10] are kinematic entities of particular interest: they provide a compact and relevant representation of fluid flows [7], they retain key information for the understanding of phenomena (e.g., vortices in study of turbulence [15], depressions in meteorology [9]), and they provide tokens for tracking purposes [10]. Such entities can be extracted a posteriori from estimated velocity fields [5, 7, 14, 16]. They can also be recovered directly from images [7, 10]. We suggest here that both types of information should be extracted in a joint and cooperative way. To this end we introduce, on the basis of a joint estimation segmentation approach [12], a coupled approach which mixes the optical flow technique proposed in [13] with an original non-linear parametric flow modeling based on vortices, sources and sinks. 2 Dense/Parametric Robust Modeling Dense motion estimation aims at estimating a velocity mapw = fws; s 2 Sg at each point of the rectangular pixel lattice S, based on brightness function f(t) = ff(s; t); s 2 Sg at two consecutive instants t and t+1. Assuming temporal constancy of the brightness function, standard optic-flow estimation rely on a differential equation known as opticflow constraint equation (OFCE): rf(s; t)Tws+ft(s) = 0 where rf stands for the spatial gradient of f and ft(s;ws) 4 = f(s; t + 1) f(s; t) denotes the finite difference approximation of the temporal derivative . This equation issues from a linearization of the brightness constancy assumption. It may also be seen as the material derivative of f (i.e., the rate of change of f as observed when moving with point s). The OFCE being known to be not valid in general for large displacements (the linearity domain of the luminance function is drastically reduced for long range displacements, as well as at sharp edge locations) an incremental version of this equation is usually considered. This technique which may be related to non-linear least squares Gauss-Newton method [1, 11] is generally used in combination with a standard multiresolution setup [2, 6]. In the following, we shall assume to work at a given resolution of such a pyramidal structure. However, one has to keep in mind that the expressions and computations are meant to be reproduced at each resolution level according to a coarseto-fine strategy. Let us now assume that a rough estimate w = fws; s 2 Sg of the unknown velocity field is available (e.g., from an estimation at lower resolution or from a previous estimation). Based on a linearization of the constancy brightness assumption from time t to t+1 aroundw, a small increment field dw 2 (R R)S can be estimated as: argmin dw H1(dw; f;w) + H2(dw;w); (1) with [2, 11]: H1 4 = Xs2S 1[rf(s+ws; t+1)Tdws+ft(s;ws)]; (2) H2 4 = X 2C 2 [k(ws + dws) (wr + dwr)k] ; (3) where > 0, C is the set of neighboring site pairs lying on grid S equipped with some neighborhood system , ft(s;ws) 4 = f(s+ws; t+1) f(s; t) is now the displaced frame difference, and 1 and 2 are standard robust M estimators (with hyper-parameters 1 and 2). Such functions penalize the deviations both from the data model (i.e., the OFCE) and from the first-order smoothing prior. The dense estimator (1-3) is general; it is only based on the assumptions of luminance conservation (first term) and of spatial smoothness of the velocity (second term). It does not rely on any prior knowledge about typical fluid flows. In most situations, it is relevant to consider that fluid motion is composed of three parts: a smooth laminar component, a divergence-free component stemming from a few Figure 1. Example of vortex (roti > 0), source (divi > 0), and translated swirl with shear (ai; bi; roti; divi and sheari 4 =p(ci fi)2 + (ei + di)2 > 0). vortices, and an irrotational component produced by a few sinks/sources. Vortices correspond to localized concentrations of vorticity rotw 4 = ux uy, whereas sinks and sources are associated to analog concentrations of divergence divw 4 = ux + vy. Extending Rankine vortex model [14], we introduce an original unified modeling of these entities. Let a vortex/sink/source be located at si = (xi; yi). In a certain neighborhood of si, the velocity field is approximated by a linear model. Beyond this neighborhood, the same linear expression is kept, but scaled by the inverse of the squared distance to si. Assuming a circular neighborhood Di of radius ri around si, we thus consider the following parametric velocity field: wi(s) 4 = min 1; r2 i ks sik2 : ai bi + ci di ei fi x xi y yi ; for s = (x; y) 2 S: (4) One can verify that the divergence and the vorticity of this field decrease as ks sik 2 beyond Di, and they respectively amount to divi 4 = ci + fi and roti 4 = ei di within this disk. Vortices correspond to significantly non-zero values of roti. Significant positive (resp. negative) values of divi correspond to sources (resp. sinks). Both situations can be combined within swirls. See examples in Figure 1. If K vortices/sinks/sources are present, the total field results from the sum of all wi’s with some laminar flow which “transports” them. We make interact these different modeling ingredients with the dense field through a robust goodness-of-fit cost function: H3(dw;wlam; 1 K ;w) 4 = Xs 3[kws + dws (wlam(s) + K Xi=1wi(s))k] (5) where wlam denotes the laminar part of the flow and i 4 = (si; ri; ai; bi; ci; di; ei; fi)T gathers the parameters relative to the ith vortex/sink/source.